Skip to content

Export LLM documentation revamp #12381

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 3 commits into
base: export-llm-docs
Choose a base branch
from
Open

Export LLM documentation revamp #12381

wants to merge 3 commits into from

Conversation

jackzhxng
Copy link
Contributor

@jackzhxng jackzhxng commented Jul 10, 2025 •
edited
Loading

Summary

Structure:

  • New Getting Started page
  • AOT (export)
    • Old getting started page, which was the NanoGPT tutorial, is moved to export-custom-llm.md, with the runner sections removed to add to the run-with-c-plus-plus.md
    • New export-llm.md page for exporting LLMs with export_llm API
  • Runtime
    • iOS/Android app docs remain, they detail steps to take after the .pte is generated for running on-device
    • Added a C++ runner page for @larryliu0820 to fill out with the new runner APIs
    • Since the QNN Llama tutorial is highly custom, we are going to leave the export section in it as well instead of dividing like we did for the rest of the tutorials

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Jul 10, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/12381

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures

As of commit 1a3e422 with merge base dd4488d (image):

NEW FAILURES - The following jobs have failed:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 10, 2025
@github-actions GitHub Actions
Copy link

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

@jackzhxng jackzhxng force-pushed the jz/export-llm-docs branch 20 times, most recently from 89e52e9 to 0ffe50a Compare July 11, 2025 22:13
@jackzhxng jackzhxng requested a review from larryliu0820 July 11, 2025 22:15
@jackzhxng jackzhxng force-pushed the jz/export-llm-docs branch from 0ffe50a to dc267a8 Compare July 14, 2025 18:36
@jackzhxng jackzhxng changed the base branch from main to export-llm-docs July 14, 2025 18:36
@jackzhxng jackzhxng marked this pull request as ready for review July 14, 2025 18:36
mergennachin
mergennachin reviewed Jul 14, 2025
View reviewed changes
@jackzhxng jackzhxng force-pushed the jz/export-llm-docs branch 4 times, most recently from 8753f2b to 6af121b Compare July 14, 2025 23:42
jackzhxng
jackzhxng commented Jul 16, 2025
View reviewed changes
Copy link
Contributor Author

@jackzhxng jackzhxng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding the runner doc @larryliu0820

metascroy
metascroy reviewed Jul 16, 2025
View reviewed changes
metascroy
metascroy reviewed Jul 16, 2025
View reviewed changes
metascroy
metascroy reviewed Jul 16, 2025
View reviewed changes
@shoumikhin
Copy link
Contributor

Please make a placeholder for ObjC/Swift runtime APIs and i'll fill it out

@jackzhxng jackzhxng force-pushed the jz/export-llm-docs branch from 6fda1ef to 1b2fd42 Compare July 16, 2025 23:23
@jackzhxng
Copy link
Contributor Author

@shoumikhin I have left the existing llama-demo-ios.md untouched and linked to it from getting-started.md, feel free to make any changes to llama-demo-ios.md that you need

mergennachin
mergennachin approved these changes Jul 17, 2025
View reviewed changes
Copy link
Contributor

@mergennachin mergennachin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

export-llm is a great page, thank you for working on this

@shoumikhin
Copy link
Contributor

Instead of llama-demo-ios.md we should probably just point to examples/demo-apps/apple_ios/LLaMA/README.md?
I meant some place in the docs where we describe the LLM runtime API in C++ and how to use it. I can append some info on ObjC/Swift API there.

@jackzhxng jackzhxng force-pushed the jz/export-llm-docs branch from 46cfe37 to 1a3e422 Compare July 19, 2025 00:21
@jackzhxng
Copy link
Contributor Author

@shoumikhin sure, feel free to open a PR against this!

shoumikhin
shoumikhin reviewed Jul 19, 2025
View reviewed changes
docs/source/index.md
- [Running with C++](llm/run-with-c-plus-plus.md)
- [Running on Android (XNNPack)](llm/llama-demo-android.md)
- [Running on Android (QNN)](llm/build-run-llama3-qualcomm-ai-engine-direct-backend.md)
- [Running on iOS](llm/llama-demo-ios.md)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you create a placeholder page like that similar to Running with C++ one?
I'll make it refer llama-demo-ios.md which seems to redirect to the readme already.

Suggested change
- [Running on iOS](llm/llama-demo-ios.md)
- [Running on iOS](llm/run-on-ios.md)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@shoumikhin shoumikhin shoumikhin left review comments

@metascroy metascroy metascroy left review comments

@mergennachin mergennachin mergennachin approved these changes

@larryliu0820 larryliu0820 Awaiting requested review from larryliu0820

@JacobSzwejbka JacobSzwejbka Awaiting requested review from JacobSzwejbka

@GregoryComer GregoryComer Awaiting requested review from GregoryComer

@lucylq lucylq Awaiting requested review from lucylq

Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. release notes: none Do not include this in the release notes
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@jackzhxng @shoumikhin @mergennachin @larryliu0820 @metascroy @facebook-github-bot